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In 2011, the International Association for K-12 Online Learn¬ 
ing released the second iteration of the National Standards 
for Quality Online Courses. These standards have been used 
by numerous institutions and states around the country to 
help design and create K-12 online courses. However, there 
has been no reported research on the validity of the standards 
or the accompanying rubric. This study compares all ele¬ 
ments under the five main standards to contemporary K-12 or 
higher education online course literature. The research con¬ 
cludes with suggested changes and additions, as well as an 
explanation as to how the research connects to a larger study 
on K-12 online course design. 


INTRODUCTION 

There are a variety of popular standards that designers can look to when 
creating an online course. The Virtual High School (VHS) collaborative, 
for example, created the NetCourse Evaluation Board in 1998 to reinforce 
the designs coming out of their 25-week graduate level course (Kozma, 
1998). In 2003, work began on the original Quality Matters (QM) rubric, 
which used a peer-review process carried out by certified QM experts 
(QM, 2014). The Southern Regional Education Board (SREB) unveiled 
standards in 2006, although the release of these standards did not describe 
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any specific process on how the standards were developed (SREB, 2006). 
One year later, iNACOL released its own standards, largely based on the 
SREB rubric, as well as the organization’s involvement with the Partner¬ 
ship for 21st Century Skills initiative (North American Council for Online 
Learning, 2007). 

This article focuses on the first stage in a larger effort to validate the iN¬ 
ACOL National Standards for Quality Online Courses in regards to online 
course design. The individual standards - as well as the processes behind 
their development - were all considered as the basis for this study. Howev¬ 
er, it was decided that this research should be based on popular, current, and 
non-proprietary standards to allow for the greatest impact on the field. In 
this article, we examine the initial development of the iNACOL standards. 
This examination is followed by a systematic discussion of each aspect of 
the iNACOL standards and whether there is research literature in the field of 
K-12 online learning, and to a lesser extent the larger field of online learn¬ 
ing. Finally, suggestions are provided with the goal of improving the stan¬ 
dards. 


LITERATURE REVIEW 

The most recent and some of the most widely used national standards on 
course design in K-12 online learning are those from iNACOL (Barbour & 
Adelstein, 2013a). Originally released in 2007, the standards were used to 
create a 0-4 point scale rubric in five areas (i.e., content, instructional de¬ 
sign, student assessment, technology, and course evaluation and support). 
Each of these five sections is further divided into multiple subsections. 
Under each subsection, the rubric provides specific elements to answer the 
overlying question, “To what extent does the course meet the criteria in this 
area?” (iNACOL, 2011, p. 8). The iNACOL National Standards for Quality 
Online Courses are a widely used design instrument currently implemented 
around the country (Barbour, Clark, DeBruler, & Bruno, 2014). For exam¬ 
ple, California, Michigan and Texas have selected the iNACOL standards 
for their statewide online initiatives (iNACOL, 2015; Michigan Department 
of Education & Michigan Virtual University, 2015). State law in Michigan 
(i.e., section 2If) allows K-12 students to enroll in online courses, and on¬ 
line courses deposited in the statewide catalog provided by Michigan dis¬ 
tricts must be reviewed against the current iNACOL standards (Michigan 
Virtual University, 2016). 

Following the release of their quality online course design standards in 
2007, iNACOL began the process of updating this initial effort by utilizing 
feedback from different organizations on the original standards (iNACOL, 
2011). Updates continued from a process of review work completed by the 
California Learning Resource Network and the Texas Agency’s Texas Virtu- 
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at School Network as they utilized the original standards to evaluate online 
course content (Smith, Bridges, & Lewis, 2013). In addition to these efforts, 
iNACOL also reconvened an expert panel in the areas of course develop¬ 
ment, instructional design, professional development, research, education, 
and administration (iNACOL, 2011). The original standards were eventu¬ 
ally updated in 2011 based on feedback from these various efforts. 

Despite the popularity of the current iNACOL National Standards for 
Quality Online Courses, there has been no research published that reports 
the validity of the standards or the published rubric that measures those 
standards (Barbour, 2013). The validation process is often begun through a 
basic literature review to examine the support the standards enjoy in the re¬ 
search, work that Ferdig, Cavanaugh, DiPietro, Black, and Dawson (2009) 
undertook with the iNACOL National Standards for Quality Online Teach¬ 
ing. However, as noted by Ferdig and his colleagues, the literature available 
was limited due to the fact that most research was about traditional class¬ 
rooms and not online courses. Further, Barbour and Reeves (2009) indicated 
that there was a much greater base of literature focused on adult popula¬ 
tions, as opposed to the K-12 environment. 

To date, one of the only research-based initiatives examining the qual¬ 
ity of online course content has been the QM program. The original QM 
standards, which focused on higher education and included 40 specific 
standards grouped under eight general standards (Legon & Runyon, 2007), 
were developed through a U.S. Department of Education’s Fund for the Im¬ 
provement of Postsecondary Education grant (Shattuck, 2007). These higher 
education standards have been supported by a full review of the published 
research literature in post-secondary education (Shattuck, 2013). 1 The ru¬ 
bric associated with these standards has been utilized in hundreds of thou¬ 
sands of instances, and has been tested for reliability and validity (Shattuck, 
2015a; Shattuck, Zimmerman, & Adair, 2014). In 2010, QM partnered with 
the Florida Virtual School to develop and begin testing the reliability and 
validity of a K-12 version of their standards and rubric (QM, 2016), which 
included its relationship to K-12 research (Shattuck, 2015b) and the existing 
iNACOL standards (QM, 2015). While QM’s annual subscription fee is be¬ 
yond the fiscal resources of many K-12 programs, the process that they have 
undertaken to validate their standards has not be replicated by any other set 
of online learning standards. All of this begs the question, are the iNACOL 
National Standards for Quality Online Course supported by existing re¬ 
search? 


1 See https://www.qmprogram.org/qmresources/research/ for a complete listing of research related 
to each individual standard. 
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METHODOLOGY 

The current study examines the construct validity of the iNACOL (2011) 
National Standards for Quality Online Courses using contemporary re¬ 
search. Validity attempts to answer the question, “Does the assessment mea¬ 
sure what it was intended to measure?” (Jonsson & Svingby, 2007, p. 136). 
More specifically, content validity attempts to show how elements of an as¬ 
sessment are relevant and representative to the construct being measured 
(Haynes, Richard, & Kubany, 1995). It has been argued that content validity 
can be determined in a variety of ways, such as a logical study of content or 
the use of quantitative scores (Fitzpatrick, 1983). In this instance, contem¬ 
porary research is compared to the rubric associated with the iNACOL stan¬ 
dards to determine if support for each of the standard elements exists within 
the research literature. 

Contemporary research was collected through Wayne State University’s 
library and connected databases, including Education Resources Information 
Center, Education and Information Technology Digital Library 2 (EdITLib), 
ProQuest, and Google Scholar. Wayne State University faculty and other rec¬ 
ommended scholars were also consulted to identify relevant and related lit¬ 
erature. Numerous search terms were used that included, but were not lim¬ 
ited to: K-12, online learning, online design, virtual school, course design 
standards, and e-learning. As research regarding K-12 online course design 
has been somewhat limited over the years (Barbour, 2013), often with a fo¬ 
cus on individual programs, the search included K-12 online learning litera¬ 
ture that was both research-based and also not based on research. In some 
instances, the search was expanded to include online learning with adult 
populations when there was a lack of K-12 research available (this was of¬ 
ten with a specific focus on the individual element). Given the number of 
elements contained in the iNACOL National Standards for Quality Online 
Courses, the goals were to find 1) two to three supporting pieces of K-12 
online learning research, 2) two to three supporting pieces of K-12 online 
learning literature, 3) two to three supporting pieces of online learning re¬ 
search, or 4) some combination of the previous items. 

RESULTS 

To answer the guiding question of validity, each of the standards from 
five areas of the iNACOL rubric were reviewed using the same format. 
Each section begins with a brief overview of the standard. Immediately fol¬ 
lowing is a table that lists the subsections with their individual elements 
linked to the associated citation(s). Following the table, each of the ele¬ 
ments are discussed in relation to the contemporary research. 

2 In February of 2016, EdITLib, was renamed The Learning and Technology Library 
(LearnTechLib.) 
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Section A: Content 

“The course provides online learners with multiple ways of 
engaging with learning experiences that promote their mastery 
of content and are aligned with state or national content stan¬ 
dards'” (iNACOL, 2011, p. 8). 

Section A of the iNACOL course design standards contained four 
sub-sections, which included 13 elements. 

Table 1 


Academic Content Standards and Assessments 

The goals and objectives clearly state what the participants will know or be able to do at the end of the 
course. The goals and objectives are measurable in multiple ways 

K-12 Literature 

Adult Population Literature 

Barbour (2007) 

Morris (2002) 

Yamashiro & Zucker (1999) 


The course content and assignments are aligned with the state's content standards, common core 
curriculum, or other accepted content standards set for Advanced Placement courses, technology, 
computer science, or other courses whose content is not included in the state standards. 

K-12 Literature 

Adult Population Literature 

Fulton (2002) 

Porter, McMaken, Hwang, & Yang (2011) 


The course content and assignments are of sufficient rigor, depth and breadth to teach the standards 
being addressed. 

K-12 Literature 

Adult Population Literature 

Thomson (2010) 

Anderson (2004) 

Information literacy and communication skills are incorporated and taught as an integral part of the 
curriculum. 

K-12 Literature 

Adult Population Literature 

Morris (2002) 

American Management Association (2012) 

Multiple learning resources and materials to increase student success are available to students before 
the course begins. 

K-12 Literature 

Adult Population Literature 

Elbaum, McIntyre, & Smith (2002) 

McKenzie, Perini, Rohlf, Toukhsati, Conduit, & 
Sanson (2013) 
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Table 1, Continued 


Course Overview and Introduction 

A clear, complete course overview and syllabus are included in the course. 

K-12 Literature 

Adult Population Literature 

Barbour (2007) 

Zucker & Kozma (2003) 


Course requirements are consistent with course goals, are representative of the scope of the course 
and are clearly stated. 

K-12 Literature 

Adult Population Literature 

Elbaum et al. (2002) 

Zucker & Kozma (2003) 


Information is provided to students, parents and mentors on how to communicate with the online 
instructor and course provider. 

K-12 Literature 

Adult Population Literature 

DiPietro, Ferdig, Black, & Preston (2008) 

Fulton (2002) 

Morris (2002) 


Legal and Acceptable Use Policies 

The course reflects multi-cultural education, and the content is accurate, current and free of bias or 
advertising. 

K-12 Literature 

Adult Population Literature 

Fulton (2002) 

Hernandez (2005) 


Expectations for academic integrity, use of copyrighted materials, plagiarism and netiquette (Internet 
etiquette) regarding lesson activities, discussions, and e-mail communications are clearly stated. 

K-12 Literature 

Adult Population Literature 

DiPietro et al. (2008) 

Elbaum et al. (2002) 

King, Guyette, & Piotrowski (2009) 

Privacy policies are clearly stated. 

K-12 Literature 

Adult Population Literature 

Children's Online Privacy Protection Act (1998) 

Family Educational Rights and Privacy Act (2011) 

Micheti, Burkell, & Steeves (2010) 
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Table 1, Continued 


Instructor Resources 

Online instructor resources and notes are included. 

K-12 Literature 

Adult Population Literature 

Davis (2003) 

Morris (2002) 

Zucker & Kozma (2003) 


Assessment and assignment answers and explanations are included. 

K-12 Literature 

Adult Population Literature 

Zucker & Kozma (2003) 

Roby, Ashe, Singh, & Clark (2013) 


Subsection: Content Standards and Assessments. Section A began 
with clearly stated goals and objectives, noting that both should be well-de¬ 
fined with multiple means of measurement. This was consistent with the ad¬ 
vice from several studies in primary and secondary distance education. For 
example, in his book discussing the design of Wichita public schools online 
program, Morris (2002) advised that, to start, every teacher should create 
an orientation video that discusses assignments, due dates, expectations, and 
many additional items. The information was posted and available to stu¬ 
dents throughout their time in the course. Similarly, Yamashiro & Zucker 
(1999) reported a panel review of online courses delivered by the VHS, 
which focused on ensuring that “benchmarks and models of performance 
[were] provided and made up front” (p. 13). Barbour’s (2007) interviews 
with course developers from the Center for Distance Learning and Innova¬ 
tion (CDL1) reinforced the importance of clear instructions and expectations 
for the students by naming the concept as one of his seven principles of cre¬ 
ating effective web-based content. 

The goals and objectives should also be aligned to state and common 
core standards, as well as other relevant sets of standards not necessarily 
included by the states, such as Advanced Placement and technology class¬ 
es. Fulton (2002) suggested that alignment with state standards is one of a 
handful of traditional indicators that could help policymakers evaluate the 
quality of online courses. There are also other reasons to consider standards 
alignment. For example, the introduction of Common Core State Standards, 
as explained by Porter et al. (2011) could bring K-12 schools shared expec¬ 
tations for all students, a greater focus on core areas as seen in international 
curriculum, allowance of states to focus on other areas in local education, 
and, possibly, improvement of the quality of common assessments. 
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After alignment, the rigor, depth, and breadth of assignments are re¬ 
viewed. There does not have to be a drastic shift from what works in tra¬ 
ditional classrooms. Teachers interviewed by Thomson (2010) believed 
online and traditional setting content could be similar, but it was how stu¬ 
dents interacted with the material that would differ, specifically noting self- 
motivation as a barrier. This was further enforced by Anderson (2004), who 
theorized that interaction was what needed to be considered to ensure depth 
of learning, noting that “sufficient levels of deep and meaningful learning 
can be developed as long as one of the three forms of interaction (student- 
teacher; student-student; student-content) is at very high levels” (p. 66). 

High levels of interaction directly tie into the importance of communica¬ 
tion and information literacy skills. This notion is not necessarily new, as 
Morris (2002) required the inclusion of a communication area for the Wich¬ 
ita online program, noting it was important for both the student and teacher 
to understand expected communication responsibilities. This element also 
had strong support not just in K-12 education, but in the business world 
as well. For example, the American Management Association (2012) was 
a strong advocate for communication and information literacy skills being 
taught in the classroom. Companies surveyed noted that critical thinking, 
communication, collaboration, and creativity were required of graduates en¬ 
tering the workforce. 

The first subsection concluded with discussing student access to re¬ 
sources before the course even begins, a notion with support in both K-12 
and higher education research. Advanced information, however, can come 
in a variety of formats, all with their own unique advantages. For an on¬ 
line course, it is appropriate to share out hardware requirements, resources, 
dates, times, and policies (Elbaum et ah, 2002). It would also be beneficial 
to offer pre-class activities, allowing for a grasp of the topic before it is 
even discussed (McKenzie et ah, 2013), which can lead to a deeper under¬ 
standing. 

Subsection: Course Overview and Introduction. Successful designers 
understand the importance of a clear syllabus and clearly defined course re¬ 
quirements that are in line with course goals. The VHS review board took 
this position, as it considered the syllabus a characteristic of a ‘high quality’ 
rated course (Zucker & Kozma, 2003). Elbaum et ah (2002) recommended 
the designer first list course objectives and then follow with activities and 
learning cycles built around the objectives. This method would guarantee 
the syllabus to line up with course goals so students know what is coming 
their way. This specific method of design neatly falls in line with the first 
of Barbour’s (2007) seven principles of creating effective web-based con¬ 
tent for adolescent learners, which were developed based on interviews with 
asynchronous course content designers. 
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Keeping with delivering information, a course should indicate how com¬ 
munication between the students, parents, instructor, mentors, and course 
provider is managed. A strong push for clearly defined communication 
can be found at the K-12 level (DiPietro et al., 2008; Fulton, 2002; Mor¬ 
ris, 2002). For example, the teachers interviewed by DiPietro et al. (2008) 
mentioned the importance of not just communication, but making use of a 
variety of channels (e.g., phone calls, email, and instant messaging). It was 
important for the teachers to have multiple ways for the students to connect 
with them and provide support when needed. 

Subsection: Legal and Acceptable Use Policies. The third subsection 
considered a handful of issues that revolve around legislative regulations 
that would impact an online course, beginning with equality in the class¬ 
room. A successful online course will respect multicultural education, al¬ 
lowing for equal learning opportunities while keeping the content up to date 
and free of bias. This element has two distinct parts, the first of which is 
making sure that all students have access to the same learning opportunities. 
As noted by Fulton (2002), “any virtual school - public or private - that ac¬ 
cepts public funding must guarantee that it does not discriminate by race, 
ethnicity, gender, disability, religion, or other categories protected by law” 
(p. 24). This can come in the form of state or federal regulations regard¬ 
ing educational equality (Flernandez, 2005). The second part of this element 
dealt with bias in the classroom, an important topic for designers to keep 
in mind. The very nature of an online course means geographic boundaries 
can become inconsequential, allowing for students with a variety of back¬ 
grounds to partake in the class. 

The next element called for a code of conduct for the class. It should 
touch upon netiquette, plagiarism, and overall academic integrity. While the 
benefits of sending out policies to the students have been previously men¬ 
tioned (Elbaum et al., 2002), teachers from the DiPietro et al. (2008) study 
specifically noted including a code of conduct and continuous monitoring of 
online behaviors. A specific code that outlines the boundaries of academic 
integrity can help in setting a proper tone for the course (King et al., 2009). 

Related to a code of conduct, privacy policies should also be posted for 
students. Laws such as the Children’s Online Privacy Protection of 1998 
and the Family Educational Rights and Privacy Act of 2011 were designed 
to protect student information, and online courses need to adhere to these 
guidelines. However, it can be difficult to explain this to a young student, 
as policies written at advanced reading levels hinder the student from un¬ 
derstanding their rights. It is no surprise, then, that children and teenagers 
prefer policies to be short, simple, and concise (Micheti et al., 2010). This 
is certainly not to say that privacy policies cannot be detailed; they just need 
to be clear, listing out the topics in a logical order. 
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Subsection: Instructor Resources. The final subsection of Section A 
looked to assure that the instructor had access to resources to help with the 
learning management system (LMS), as well as built-in course assessments, 
answers, and explanations. Unsurprisingly, support for courses in the form 
of design and material are important to educators in general (Roby et ah, 
2013), therefore, they have become a high priority in many K-12 online 
programs (Davis, 2003; Morris, 2002; Zucker & Kozma, 2003). Specific 
training within the LMS itself has helped Michigan Virtual School (MVS) 
and VHS educators gain a practical knowledge about their online space, al¬ 
lowing for opportunities to create resources educators will come to depend 
on (i.e., assessments and answers - Davis, 2003; Zucker & Kozma, 2003). 
For example, this was the rationale for the process of resource creation that 
was actually mandated as part of the Teachers Learning Conference, a re¬ 
quired course for all VHS educators (Zucker & Kozma, 2003). 

Section B: Instructional Design 

“The course uses learning activities that engage students in 
active learning; provides students with multiple learning paths 
to master; the content is based on student needs; and provides 
ample opportunities for interaction and communication — stu¬ 
dent to student, student to instructor and instructor to student’’ 
(iNACOL, 2011, p. 11). 

Section B of the iNACOL course design standards contained five subsec¬ 
tions, which included 11 elements. 


Table 2 


Instructional and Audience Analysis 

Course design reflects a clear understanding of all students' needs and incorporates varied ways to 
learn and master the curriculum. 

K-12 Literature 

Adult Population Literature 

DiPietro et al. (2008) 

Kapitzke & Pendergast (2005) 

Looi, Zhang, Chen, Seow, Chia, Norris, & Soloway (2011) 

Simpson & Park (2013) 
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Table 2, Continued 


Course, Unit and Lesson Design 

The course is organized by units and lessons that fall into a logical sequence. Each unit and lesson 
includes an overview describing objectives, activities, assignments, and resources to provide multiple 
learning opportunities for students to master the content. 

K-1 2 Literature 

Adult Population Literature 

Barbour (2007) 

Barbour & Adelstein (2013b) 

DiPietro et al. (2008) 


Instructional Strategies and Activities 

The course instruction includes activities that engage students in active learning. 

K-1 2 Literature 

Adult Population Literature 

Barbour & Adelstein (2013b) 

Selco, Bruno, & Chan (2012) 

Chen, Lambert, & Guidry (2010) 

Hoic-Bozic, Mornar, & Boticki (2009) 

The course and course instructor provide students with multiple learning paths, based on student 
needs that engage students in a variety of ways. 

K-1 2 Literature 

Adult Population Literature 

Christensen, Horn, & Johnson (2011) 

Horn & Stalker (2015) 

Packard (2013) 

VanderArk (2012) 


The course provides opportunities for students to engage in higher-order thinking, critical reasoning 
activities and thinking in increasingly complex ways. 

K-1 2 Literature 

Adult Population Literature 

Murphy, Rowe, Ramani, & Silverman (2014) 


The course provides options for the instructor to adapt learning activities to accommodate students’ 
needs. 

K-1 2 Literature 

Adult Population Literature 

Christensen, Horn, & Johnson (2011) 

Horn & Stalker (2015) 

Mastropieri, Scruggs, Norland, Berkeley, McDuffie, Torn- 
quist, & Connors (2006) 

Packard (2013) 

VanderArk (2012) 


Readability levels, written language assignments and mathematical requirements are appropriate for 
the course content and grade-level expectations. 

K-1 2 Literature 

Adult Population Literature 

Barbour (2007) 

DiPietro et al. (2008) 
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Table 2, Continued 


Communication and Interaction 

The course design provides opportunities for appropriate instructor-student interaction, including op¬ 
portunities for timely and frequent feedback about student progress. 

K-12 Literature 

Adult Population Literature 

DiPietro et al. (2008) 

Reeves, Vangalis, Vevera, Jensen, & Gillan (2007) 


The course design includes explicit communication/activities (both before and during the first week of 
the course) that confirms whether students are engaged and are progressing through the course. The 
instructor will follow program guidelines to address non-responsive students. 

K-12 Literature 


Johnston & Barbour (2013) 


The course provides opportunities for appropriate instructor-student and student-student interaction to 
foster mastery and application of the material. 

K-12 Literature 

Adult Population Literature 

Elbaum et al. (2002) 

Rice (2012) 


Resources and Materials 

Students have access to resources that enrich the course content. 

K-12 Literature 

Adult Population Literature 

Cavanaugh (2013) 

Elbaum et al. (2002) 

Rice (2012) 



Subsection: Instructional and Audience Analysis. The first subsection 
of B focused on understanding the needs of your students and incorporat¬ 
ing a variety of ways to learn the curriculum. Individualized instruction and 
differentiating are not new concepts in education. The ideas can be read¬ 
ily found in the K-12 online environment (DiPietro et al. 2008; Kapitzke & 
Pendergast, 2005; Looi et ah, 2011; Simpson & Park, 2013). The challenge 
was trying to discover strengths and weaknesses of each participant in a stu¬ 
dent-centered environment where interaction from the instructor was mini¬ 
mal. Success has been found when the teachers consistently monitor the 
class, which is what occurred with Michigan virtual educators (DiPietro et 
ah, 2008). By reviewing students, the educators were able to discover learn¬ 
ing styles and gaps, which was considered a best practice. 
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Subsection: Course, Unit and Lesson Design. The second subsection 
looked at the logical order of units, a posted overview outlining objec¬ 
tives, activities, and assignments, and the resources to allow multiple path¬ 
ways for student success. While there was only one element mentioned in 
the second subsection, it contained individual requirements that should be 
reviewed separately. To start, course sequencing was beneficial to both the 
student and the designer, which is why it has been viewed as a vital ped¬ 
agogical strategy for online education (DiPietro et al., 2008). It is also an 
area that was previously discussed in Section A (see “Subsection: Course 
Overview and Introduction). 

Creating an overview or summary of the lessons can be helpful for stu¬ 
dents, especially those in nontraditional online courses, where asking a 
teacher to clarify can be a drawn out process (Barbour & Adelstein, 2013b). 
When Barbour (2007) discussed the seven principles of online course de¬ 
sign, one teacher in particular noted that he created notes and examples be¬ 
cause a lot of his students “...were isolated, and knowing that they didn’t 
have access to a [content-area] teacher readily whenever they wanted... so 
I tried to make the websites... compensate for that as much as I possibly 
could” (p. 103). 

Subsection: Instructional Strategies and Activities. The first element 
suggested active learning should be considered as the course is designed. Ac¬ 
tive learning can be an important factor for student success, as it gives them 
a connection to the concepts being taught, which in turn allows for student- 
created content (Barbour & Adelstein, 2013b). Students involved with active¬ 
learning courses have readily shown higher scores on statewide exams (Selco 
et al., 2012). Active learning has also worked in higher education settings 
(Chen et al., 2010; Hoic-Bozic et al., 2009), showing a connection to higher 
order and critical thinking skills, which tied directly into the third element. 

Allowing for higher order and critical thinking is not a new concept in ed¬ 
ucation. The concern is that engagement in critical thinking is minimal when 
adolescents are left to their own devices. However, if the classroom environ¬ 
ment is set up to reinforce such behaviors, it can be promoted with the stu¬ 
dents (Murphy et al., 2014). 

Both elements two and four shared similar ideas, discussing multiple 
pathways and adaptive activities, all based on students’ individual needs. The 
concept of individualization was discussed above, which showed strong sup¬ 
port in K-12 online learning). Differentiating lessons can yield powerful re¬ 
sults (Christensen et al., 2011; Horn & Stalker, 2015; Packard, 2013; Vander 
Ark, 2012). For example, Mastropieri et al. (2006) showed that eighth grade 
science classes had comparatively higher scores on both unit and state exams 
than classes who stuck with traditional lecture and lab activities. 
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Finally, it is important to note that adapting can be more than just dif¬ 
ferentiating. The learner’s skills are taken into consideration. Understand¬ 
ing students and designing appropriate lessons that target average or below 
average students (DiPietro et al., 2008) - with extension activities for those 
on the higher end (Barbour, 2007) - will help curb confusion with the mate¬ 
rials. 

Subsection: Communication and Interaction. A key to success for on¬ 
line courses is communication. Without the advantage of face-to-face inter¬ 
actions, the course design must provide different opportunities for instruc¬ 
tor-student discussion. Frequent and prompt feedback is supported in K-12 
literature, noting that teachers should respond within a 48-hour period from 
submission of the assignment (Reeves et ah, 2007). Not only does feedback 
keep motivation levels high, but a long waiting period has the potential of 
lowering student engagement (DiPietro et ah, 2008). This is important to 
consider, as keeping students engaged for an online course can be challeng¬ 
ing. Even with high quality materials, the ability to have face-to-face de¬ 
bates, discussions, and role playing are seen as more attractive to students 
(Johnson & Barbour, 2013). 

While teachers should be involved, it is important to let the students lead 
the conversation and for teachers to not take over discussion threads (El- 
baum et ah, 2002). This ultimately can help to form an online community. 
Working towards a strong community will naturally lead to collaboration 
activities, such as blogs, video conferencing, simulations, group projects, 
and jigsaw sharing (Rice, 2012). 

Subsection: Resources and Materials. Proper resources will also help 
students foster mastery of a subject. The use of virtual manipulatives, for 
example, has garnered higher performance results in algebra courses that 
took advantage of this unique resource (Cavanaugh, 2013). This does not 
mean that traditional resources should be ignored, as hard copy materials 
can have a positive impact as well (Elbaum et ah, 2002). 

Resources can originate from multiple sources, which can be over¬ 
whelming for educators and designers not knowing where to even begin. 
Trusted sites, such as PBS or Scholastic, are an excellent place to begin the 
search (Rice, 2013). Educators should also search out teacher specific sites 
that link directly to appropriate media, simulations, and games that are read¬ 
ily available. 

Section C: Student Assessment 

“The course uses multiple strategies and activities to as¬ 
sess student readiness for and progress in course content and 
provides students with feedback on their progress” (iNACOL 
2011, p.14). 
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Section C of the iNACOL course design standards contained three sub¬ 
sections, which included seven areas of measurement. 

Table 3 


Evaluation Strategies 

Student evaluation strategies are consistent with course goals and objectives, are representative of 
the scope of the course and are clearly stated. 

K-12 Literature 

Adult Population Literature 

DiPietro (2010) 

Zucker & Kozma (2003) 


The course structure includes adequate and appropriate methods and procedures to assess students' 
mastery of content. 

K-12 Literature 

Adult Population Literature 

Naidu (2013) 

Palmer & Devitt (2014) 

Feedback 

Ongoing, varied, and frequent assessments are conducted throughout the course to inform instruction. 

K-12 Literature 

Adult Population Literature 

Cavanaugh, Barbour, & Clark (2009) 


Assessment strategies and tools make the student continuously aware of his/her progress in class and 
mastery of the content. 

K-12 Literature 

Adult Population Literature 

Naidu (2013) 

Rice (2012) 


Assessment Resources and Materials 

Assessment materials provide the instructor with the flexibility to assess students in a variety of ways. 

K-12 Literature 

Adult Population Literature 

Morris (2002) 


Grading rubrics are provided to the instructor and may be shared with students. 

K-12 Literature 

Adult Population Literature 

Hall & Salmon (2003) 

Rice (2012) 


The grading policy and practices are easy to understand. 

K-12 Literature 

Adult Population Literature 

Rice (2012) 
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Subsection: Evaluation Strategies. Successful online courses include 
student evaluation strategies that align with course objectives and are con¬ 
sistent with goals. Educators who use multiple and appropriate means of as¬ 
sessment do this for more than just keeping tabs on students. It helps engage 
students with the content (DiPietro, 2010), keeping them in lock step with 
the goals. This should all be clearly stated to the student, possibly outlined 
in the syllabus (Zucker et ah, 2003), as was previously mentioned in Section 
A. 

Evaluation strategies are only as good as the methods and procedures 
used, which have to be able to assess mastery of content. The open progres¬ 
sion of online courses can make this difficult, but designers need to imple¬ 
ment assessments that are valid, reliable, equitable, and secure (Naidu, 
2013). When implemented within online higher education courses, multiple 
means of formative and summative assessments helped students show sig¬ 
nificant improvement in mastery of the material (Palmer & Devitt, 2014). 

Subsection: Feedback. Evaluations should not necessarily be a simple 
snapshot in time. The feedback itself can also come from both the student 
and the teacher. Prior research in K-12 online courses, for example, showed 
high value in using student feedback (Cavanaugh et al., 2009). 

Students are also generally enthusiastic to hear feedback and advice on 
how to achieve mastery (Naidu, 2013), and should be a high priority for ed¬ 
ucators. The feedback should be meaningful to the understanding, as well 
as given in a timely manner (Rice, 2002). Much like the methods used, the 
feedback itself should be clear to the students and easily accessible. 

Subsection: Assessment Resources. Evaluation materials should be 
varied, allowing for multiple means of assessment. There are similar meth¬ 
ods that can be shared between online and traditional settings. These would 
include preparation materials, rubrics, and any other resources required for 
course, state, and district assessments (Morris, 2002). 

It can be argued that students should see course rubrics, as it allows them 
to see what exactly the instructor expects (Rice, 2002). Since rubrics are 
presented in a matrix format, students can make note of their own strengths 
and weaknesses (Hall & Salmon, 2003). Regardless of the assessment the 
teacher selects and their decision to share that with students, Rice com¬ 
mented that the grading policy should be specifically outlined in the course 
syllabus or frequently asked questions (FAQ) and readily available to the 
students. 
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Section D: Technology 

“The course takes full advantage of a variety’ of technol¬ 
ogy’ tools, has a user-friendly interface and meets accessibil¬ 
ity’ standards for interoperability’ and access for learners with 
special needs” (iNACOL, 2011, p. 15). 

Section D of the iNACOL course design standards contained five sub¬ 
headings, which included 11 elements. 

Table 4 


Course Architecture 

The course architecture permits the online instructor to add content, activities and assessments to 
extend learning opportunities. 

K-12 Literature 

Adult Population Literature 

Barbour, Morrison, & Adelstein (2014) 

Rice (2012) 


The course accommodates multiple school calendars; e.g., block, 4x4 and traditional schedules. 

K-12 Literature 

Adult Population Literature 

Wicks (2010) 


User Interface 

Clear and consistent navigation is present throughout the course. 

K-12 Literature 

Adult Population Literature 

Barbour (2007) 

Barbour, Morrison, & Adelstein (2014) 

Morris (2002) 


Rich media are provided in multiple formats for ease of use and access in order to address diverse 
student needs. 

K-12 Literature 

Adult Population Literature 

Barbour (2007) 

Cavanaugh (2013) 

Keeler, Richter, Anderson-lnman, Horney, & Ditson 
(2007) 
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Table 4, Continued 


Technology Requirements and Interoperability 

All technology requirements (including hardware, browser, software, etc...) are specified. 

K-12 Literature 

Adult Population Literature 

DiPietro et al. (2008) 

Elbaum et al. (2002) 


Prerequisite skills in the use of technology are identified. 

K-12 Literature 

Adult Population Literature 

Elbaum et al. (2002) 

Rice (2012) 


The course uses content-specific tools and software appropriately. 

K-12 Literature 

Adult Population Literature 

DiPietro et al. (2008) 


The course is designed to meet internationally recognized interoperability standards. 

K-12 Literature 

Adult Population Literature 

Watson & Watson (2007) 

Coates, James, & Baldwin (2005) 

Copyright and licensing status, including permission to share where applicable, is clearly stated and 
easily found. 

K-12 Literature 

Adult Population Literature 

Elbaum et al. (2002) 

Halme & Somervouri (2012) 

Tonks, Westin, Wiley, & Barbour (2013) 


Accessibility 

Course materials and activities are designed to provide appropriate access to all students. The course, 
developed with universal design principles in mind, conforms to the U.S. Section 504 and Section 508 
provisions for electronic and information technology as well as the W3C’s Web Content Accessibility 
Guidelines (WCAG 2.0). 

K-12 Literature 

Adult Population Literature 

Elbaum et al. (2002) 

Rose, Smith, Johnson, & Glick (2015) 


Data Security 

Student information remains confidential, as required by the Family Educational Rights and Privacy 

Act (FERPA). 

K-12 Literature 

Adult Population Literature 

Barbour & Plough (2012) 

Waters (2011) 

Cantrell (2013) 
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Subsection: Course Architecture. When teaching an online course, 
the instructor needs to be able to add content, activities, and assessments 
through the LMS. The LMS is an integral part of the virtual classroom, 
so it is of high importance to select the most effective architecture for the 
course (Rice, 2012). It should be a priority of the teacher to learn what the 
LMS can accomplish and to look elsewhere if elements required are missing 
(Barbour, Morrison, & Adelstein, 2014). 

The LMS and the course itself should also be able to adjust for multi¬ 
ple calendars, such as year-round, block, and traditional. Considering the 
very nature of online learning, flexibility is a major selling point for online 
courses, giving students the opportunity to work around scheduling conflicts 
(Wicks, 2010). This can be extrapolated out to the school district, allowing 
the flexibility to work within their calendar model. 

Subsection: User Interface. The user should be able to easily move 
around the online course, with a clear and consistent navigation present. 
Some successful online courses, such as those featured from the Wich¬ 
ita catalogue, offered navigational forms in the course information area 
(Morris, 2002). These forms outlined how to find specific items within 
the course. Regardless of how the information is rolled out, the navigation 
should be kept simple and consistent for the students (Barbour, 2007). This 
can be accomplished by using a template as the course is initially designed 
(Barbour, Morrison, & Adelstein, 2014). 

Besides navigation, multimedia should also offer ease of use, with mul¬ 
tiple formats available to help address student needs. This measurement is 
supported for multiple reasons. When working in a unique and unfamiliar 
environment, having a variety of media can be helpful in supporting student 
understanding (Barbour, 2007; Cavanaugh, 2013). Multiple formats can 
possibly make the content easier to access for students with complications. 
Both legislation and Internet watchdog groups have offered recommenda¬ 
tions for teachers looking to maximize media for their students (Keeler et 
al., 2007). 

Subsection: Technology Requirements and Interoperability. Al¬ 
though seemingly obvious, a review of the technology and interoperability 
of the course must take place. Teachers should consider all aspects of stu¬ 
dent access for the course during the design process (DiPietro et al., 2008). 
As the course rolls out, Elbaum et al. (2002) recommended to specify both 
the technology and the skill requirements to the students. Even basic gen¬ 
eral overviews and procedures, such as how to access a web browser, need 
to be shared before the course begins so there are no surprises for incoming 
students. A student orientation and transition period to allow students with¬ 
out the proper skillset to gain guidance and support is recommended (Rice, 
2012 ). 
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Before sharing the tools and software used, it should be understood that 
the technology in place is appropriate for the course and that it meets in¬ 
teroperability standards. Teachers should not simply use the technology just 
because it is available to them, but instead they should make instruction¬ 
al technology decisions based on the nature of the content and their peda¬ 
gogical strategies (DiPietro et ah, 2008). These decisions should all be done 
through an LMS that can communicate with other systems within the insti¬ 
tution to share data collected (Coates et ah, 2005; Watson & Watson, 2007). 

Designers and teachers alike must also be aware of copyright issues and 
understand the importance of licensing information. While it is possible to 
obtain copyright permissions (Elbaum et ah, 2002), there are very few other 
options to legally use digital media. There is also, however, a push for free 
use under the creative commons license and that open access can be a viable 
solution (Halme & Somervouri, 2011; Tonks et ah, 2013). 

Subsection: Accessibility. In the previous sections, there were numerous 
measurements reviewing multiple means of media, resource, and course ac¬ 
cess. The reason was to guarantee that the course adhered to the law, ensur¬ 
ing universal access for all. This can seem daunting at first, but there are 
free sites that can help identify problems with accessibility (Elbaum et ah, 
2002 ). 

Accessibility is not something to lightly gloss over, however, as the law 
can and will be put in effect. In 2007, for example, a school district denied 
special needs students from online courses (Rose et ah, 2015). The district 
noted that these students were not allowed to access the course due to a dif¬ 
ficulty in completing work independently, as well as having low reading and 
writing abilities. The district was eventually cited by the U.S. Department of 
Education’s Office for Civil Rights and was forced to reverse their policy. 

Subsection: Data Security. The area of measurement for Section D re¬ 
quired that the course follow the law assuring that student information is 
confidential. Originally created in 1974, FERPAmust be adhered to by most 
higher education and K-12 education institutions. As Cantrell (2013) point¬ 
ed out, FERPA protects the student from public disclosure of private and 
educational records. However, new rules complicate the law. Institutions are 
allowed to use student records in the database for various audits, such as 
evaluating student training. Students can opt out, but it appears to be an all 
or nothing (Cantrell, 2013). A student who opts out of being used in audits 
under FERPA also coidd not be highlighted in a public newsletter for re¬ 
ceiving an award. The rules are complicated, and instructors must be aware 
of the law, especially as it applies to each student. 

The issue of data security is particularly difficult when it comes to on¬ 
line courses trying to leverage the power of popular social media sites 
(e.g., Facebook, Twitter, etc.). As noted by Waters (2011), Facebook does 
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not have a separate area for education, so students are required to enter in 
personal information. Like other aspects of the Internet, social media can 
be susceptible to numerous threats. Educators should consider social media 
sites that cater to K-12 (e.g., Edmodo, Google Apps for Education, etc.). 
These sites do not require students to enter their private information, are run 
by the teacher, and are designed specifically for classroom use - allowing 
for a safer online environment. Social networks that can provide a protected 
environment can provide numerous curricular, co-curricular, and extracur¬ 
ricular opportunities for K-12 online learning (Barbour & Plough, 2012). 

Section E: Course Evaluation and Support 

“The course is evaluated regularly for effectiveness, using a 
variety of assessment strategies, and the findings are used as 
a basis for improvement. The course is kept up to date, both 
in content and in the application of new research on course 
design and technologies. Online instructors and their students 
are prepared to teach and learn in an online environment and 
are provided support during the course” (iNACOL, 2011, p. 

18). 

Section E of the iNACOL course design standards contained four 
subheadings, which included 10 elements. 

Table 5 


Accessing Course Effectiveness 

The course provider uses multiple ways of assessing course effectiveness. 

K-12 Literature 

Adult Population Literature 

Fulton (2002) 

Morris (2002) 


The course is evaluated using a continuous improvement cycle for effectiveness and the findings used 
as a basis for improvement. 

K-12 Literature 

Adult Population Literature 

Elbaum et al. (2002) 

Zucker & Kozma (2003) 


Course Updates 

The course is updated periodically to ensure that the content is current. 

K-12 Literature 

Adult Population Literature 

Ebert & Powell (2015) 
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Table 5, Continued 


Certification 

Course instructors, whether face-to-face or virtual, are certificated and “highly qualified.” The online 
course teacher possesses a teaching credential from a state-licensing agency and is “highly-qualified” 
as defined under ESEA. 

K-12 Literature 

Adult Population Literature 

Yang & Rice (2015) 


Instructor and Student Support 

Professional development about the online course delivery system is offered by the provider to assure 
effective use of the courseware and various instructional media available. 

K-12 Literature 

Adult Population Literature 

Barbour, Morrison, & Adelstein (2014) 

Cavanaugh (2013) 

Zucker & Kozma (2003) 


The course provider offers technical support and course management assistance to students, the 
course instructor, and the school coordinator. 

K-12 Literature 

Adult Population Literature 

Barbour, Kinsella, Wicks, & Toker (2009) 

Elbaum et al. (2002) 


Course instructors, whether face-to-face or virtual, have been provided professional development in 
the behavioral, social and when necessary, emotional aspects of the learning environment. 

K-12 Literature 

Adult Population Literature 

DiPietro et al. (2008) 

Roman, Kelsey, & Lin (2010) 

Course instructors, whether face-to-face or virtual, receive instructor professional development, which 
includes the support and use of a variety of communication modes to stimulate student engagement 
online. 

K-12 Literature 

Adult Population Literature 

DiPietro et al. (2008) 


The provider assures that course instructors, whether face-to-face or virtual, are provided support, as 
needed, to ensure their effectiveness and success in meeting the needs of online students. 

K-12 Literature 

Adult Population Literature 

Elbaum et al. (2002) 

Morris (2002) 


Students are offered an orientation to taking an online course before starting the coursework. 

K-12 Literature 

Adult Population Literature 

Elbaum et al. (2002) 

Rice (2012) 
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Subsection: Accessing Course Effectiveness. To properly assess effec¬ 
tiveness, the design should allow for multiple means of evaluation of the 
course itself, be it peer review, student feedback, or course evaluations. 
While students and families can be part of the evaluation process, teach¬ 
ers should also discuss their courses together, which will allow for unique 
peer feedback (Morris, 2002). Evaluations should look similar to traditional 
classrooms for some aspects (i.e., achievement, completion rates), but also 
have parts unique to online (e.g., effectiveness of technology, course design 
interactivity) (Fulton, 2002). 

However, evaluation process should not be a one-time event. A continu¬ 
ous improvement cycle should be used for effectiveness and improvements 
(Barbour, 2005). Successful virtual schools use continuous internal and ex¬ 
ternal evaluations to make sure a high standard is maintained (Zucker & 
Kozma, 2003). Post-course, anonymous feedback from the students, as well 
as peers, should be taken into consideration at the end of every course (El- 
baum et al., 2002). 

Subsection: Course Updates. Once the evaluations have been complet¬ 
ed, the course should be updated periodically to keep content current. This 
can be challenging if the educator is working with an inflexible or an out-of- 
date infrastructure. The Clark County School District (CCSD) VHS, which 
has more than 100,000 students enrolled in blended and online courses, un¬ 
derstood the importance of updating for their massive population, and ended 
up providing an excellent example for others to follow (Ebert & Powell. 
2015). The CCSD VHS overcame challenges with updating by ensuring all 
digital content was in HTML code. This allowed the design team to easily 
evaluate and change content when required. Continuously updating policies, 
content and professional development became a part of the school’s best 
practices for student and school success. 

Subsection: Certification. The subsection of certification checks that 
the instructor is both certified and highly qualified, as noted in the Elemen¬ 
tary Secondary Education Act (ESEA) of 2001. Numerous states, such as 
Idaho, take qualification a step further by offering a K-12 online teaching 
endorsement (Yang & Rice, 2015). For example, Boise State University’s 
program took the K-12 online teaching standards set forth by iNACOL, the 
International Society for Technology Education, and the National Education 
Association, as well as the highly qualified teacher standards, and created a 
competency-based program specifically for educators teaching in an online 
environment. Partnering with virtual schools and the state’s online supple¬ 
mental program, educators gain a unique hands-on experience. 

Subsection: Instructor and Student Support. The final subsection of 
the iNACOL rubric included six areas of measurement focused on instruc¬ 
tor and student support. It is imperative that professional development take 
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place for teachers (Barbour, Morrison, & Adelstein, 2014), as online courses 
require unique skills not found in traditional settings (Cavanaugh, 2013). 
The VHS program, for example, used a required 26-week online profession¬ 
al development and design course. Teachers work exclusively in the LMS to 
train and build their own course (Zucker & Kozma, 2003). 

After the professional development, continuous support is needed (Bar¬ 
bour et al., 2009). Support should be available in a variety of formats for 
both teachers and students alike. Support should also be specific to the 
unique online environment. Technical and course management help, for in¬ 
stance, can come through training, system administration, and just taldng 
the time to play within the LMS itself (Elbaum et ah, 2002). 

Other aspects to consider are preparation for behavioral, social, and emo¬ 
tional challenges of an online setting, communicating to stimulate engage¬ 
ment, as well as succeeding in the online environment. Successful teachers 
should be active in their own course to identify students in need and know 
the proper actions to take, as well as modeling and encouraging proper com¬ 
munication that is both content and non-content related (DiPietro et ah, 
2008). While some teachers are naturally adept with these techniques, these 
items can be included in teacher preparation courses. The Preparing Online 
Instructors program, for example, is a six-week online training course for 
online instructors (Roman et ah, 2010). A survey conducted of 40 instruc¬ 
tors who went through the training showed that the vast majority felt that 
the training was necessary to increase their technological skills, as well as 
their pedagogical orientations for online instruction. The Wichita online 
public schools program also created training for online instructors, which 
had teachers working in the LMS designing, as well as collaborating, with 
their peers (Morris, 2002). While support continued on after the training 
program, teachers felt proficient enough to carry on independently. Finally, 
a strong administrative team should be in place to offer support in numerous 
areas (i.e., registration, policies, training) to help ensure success (Elbaum et 
al., 2002). 

The last area of measurement promoted students being offered an orien¬ 
tation for taking an online course before the class began. As mentioned ear¬ 
lier, Rice (2012) specifically mentioned an orientation for all students to get 
them acclimated using online instruction. The need for orientations was also 
previously noted by Elbaum et al. (2002), who recommended an orientation, 
which included a welcome letter and an information packet. 
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DISCUSSION 

While the 2011 update to the iNACOL standards has support among 
contemporary literature, one area of concern was potential omissions from 
the standards. Support, assessment, and instruction were all covered by the 
standards. It should be noted that the iNACOL standards are described as 
‘national standards for quality online courses,’ and not specifically qual¬ 
ity online course design. The omission of the term ‘design’ indicated that a 
quality online course might include elements that went beyond strict online 
course design issues. For example, ‘Section E: Course Evaluation and Sup¬ 
port’ contained several elements that were inconsistent with a strict focus on 
online course design: 

• E4: Course instructors, whether face-to-face or virtual, are certificated 
and “highly qualified.” The online course teacher possesses a teaching 
credential from a state-licensing agency and is “highly-qualified” as de¬ 
fined under ESEA. 

• E5: Professional development about the online course delivery system 
is offered by the provider to assure effective use of the courseware and 
various instructional media available. 

• E7: Course instructors, whether face-to-face or virtual, have been pro¬ 
vided professional development in the behavior, social and when neces¬ 
sary, emotional aspects of the learning environment. 

• E8: Course instructors, whether face-to-face or virtual, receive instruc¬ 
tor professional development, which includes the support and use of a 
variety of communication modes to stimulate student engagement on¬ 
line. 

• E9: The provider assures that course instructors, whether face-to-face 
or virtual, are provided support, as needed, to ensure their effectiveness 
and success in meeting the needs of online students. 

• E10: Students are offered an orientation to taking an online course be¬ 
fore starting the coursework. 

With this in mind, it is important to note that the standards did not directly 
address any elements that may be included in ‘quality online courses’ re¬ 
lated to the concept of student motivation. 

McCombs and Vakili (2005) discussed the 14 Learner-Centered Psy¬ 
chological Principles (American Psychological Association, 1997), which 
were grouped into four factors: cognitive and metacognitive factors, devel¬ 
opmental and social factors, individual-differences factors, and motivational 
and affective factors. The motivational and affective domain included three 
principles: 
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“Principle 7: Motivational and emotional influences on learning 

• What and how much is learned is influenced by the learner’s motiva¬ 
tion. Motivation to leam, in turn, is influenced by the individual’s emo¬ 
tional states, beliefs, interests and goals, and habits of thinking. 

Principle 8: Intrinsic motivation to learn 

• The learner’s creativity, higher order thinking, and natural curiosity all 
contribute to motivation to learn. Intrinsic motivation is stimulated by 
tasks of optimal novelty and difficulty, relevant to personal interests, and 
providing for personal choice and control. 

Principle 9: Effects of motivation on effort 

• Acquisition of complex knowledge and skills requires extended learner 
effort and guided practice. Without learners’ motivation to learn, the will¬ 
ingness to exert this effort is unlikely without coercion,” (p. 1585). 

Tying these principles to K-12 online education, the authors recognized the 
connection between online learning and self-directed learners, a connection 
that is made through motivational strategies. 

This was further supported by Cheng and Jang (2010), who mentioned 
in their research that motivation was an integral part of education. Using a 
self-determination theory as a way to view motivation, their study highlight¬ 
ed that the perceived satisfaction in autonomy, relatedness, and competency 
directly affected student motivation. The study also suggested understand¬ 
ing why a student was taking the course and to use the information for mo¬ 
tivation. Once again, the perceived interactions were important to student 
satisfaction. Further, Kim, Park, and Cozart (2014) also found a connection 
between self-efficacy, emotions, and motivation in their study of 72 online 
high school students in a mathematics course. Results showed how different 
emotions of the students impacted overall learning, with anger, boredom, 
and enjoyment significant predictors of achievement. If the iNACOL Na¬ 
tional Standards for Quality Online Courses go beyond a strict focus on on¬ 
line course design, elements related to student motivation are conspicuously 
absent. 


CONCLUSIONS AND IMPLICATIONS 

The five sections of the iNACOL National Standards for Quality On¬ 
line Courses were reviewed in detail. The elements were aligned to cur¬ 
rent literature in an attempt to begin the process of validating these stan¬ 
dards - a process that has never been undertaken, even though the standards 
have been widely adopted by schools, districts, and even several states. 
The results indicated the elements did align. For example, ‘Section A: 
Content’ as a whole aligned with current literature. While the subsection 
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‘Course Overview’ and ‘Introduction’ aligned with solely K-12 literature, 
‘Academic Content Standards and Assessments,’ ‘Legal and Acceptable Use 
Policies,’ and ‘Instructor Resources’ were supplemented with adult popula¬ 
tion literature. 

‘Section B: Instructional Design’ found connections to K-12 literature at 
a more consistent pace than Section A. Subsections ‘Instructional and Audi¬ 
ence Analysis,’ ‘Course, Unit,’ and ‘Lesson Design,’ ‘Communication and 
Interaction,’ and ‘Resources and Materials’ were all strongly supported by 
K-12 literature. Only the subsection related to ‘Instructional Strategies and 
Activities’ required the use of adult population literature for additional sup¬ 
port of specific elements. ‘Section C: Student Assessment’ contained three 
subsections, all of which were strongly supported by K-12 literature. The 
subsections on ‘Feedback’ and ‘Assessment Resources and Materials’ sole¬ 
ly used K-12 material in relation to the elements. Only the first subsection 
(i.e., Evaluation Strategies) relied on adult population literature for supple¬ 
mental support. 

‘Section D: Technology’ was supported mainly by K-12 literature. How¬ 
ever, subsections on ‘Technology Requirements and Interoperability’ and 
‘Data Security’ did require supplemental adult population literature for sup¬ 
port. The other subsections were all fully supported by K-12 literature for 
each element. Finally, ‘Section E: Course Evaluation and Support’ was sup¬ 
ported by K-12 literature, with the exception of one element from the sub¬ 
section related to ‘Instructor and Student Support. ’ While the literature into 
K-12 online learning course design is still developing, most elements were 
supported or supported somewhat by K-12 online learning literature, al¬ 
though not necessarily K-12 online learning research. Those elements only 
somewhat supported found additional alignment with broader online learn¬ 
ing literature related to adidt populations. 

As noted above one of the main limitations of this attempt to achieve 
the content validity of the iNACOL National Standards for Quality Online 
Courses was the lack of literature, and in particular the lack of research, 
related to K-12 online course design (Barbour & Reeves, 2009). When at¬ 
tempting to supplement with adult population literature, the challenge was 
trying to locate appropriate higher education literature with a search focused 
primarily on K-12. This, in turn, limited the scope of higher education re¬ 
search used. A final issue was that of length of the manuscript. Even when 
the editors of the Journal of Online Learning Research graciously allow for 
a greater word limit utilizing their online format, the authors still needed to 
take overall length into consideration when describing the literature support 
for each element. The iNACOL standards contain 52 total elements, which 
only allowed for a cursory review to be presented in this manuscript. 

However, in the process of examining standards in relation to the litera¬ 
ture there appeared to be some redundancy in the elements. It also became 
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clear that certain elements could be considered for consolidation as this lit¬ 
erature review occurred. Further, the literature indicated that student moti¬ 
vation was directly tied to student support and satisfaction. Flowever, while 
the current standards implied a need for motivational elements (e.g., satis¬ 
faction), there was not a clearly identified standard to examine criteria for 
motivation. 

The 2011 iNACOL National Standards for Quality Online Courses cov¬ 
er a wide breadth of topics for K-12 online courses. The literature review 
and accompanying suggestions were an important first step, but further re¬ 
search is required. For example, a more comprehensive review of the stan¬ 
dards through the lens of K-12 online literature would be useful given the 
constraints of length. The review of each element is much briefer than what 
could have been done without space considerations. Within this large scope 
of elements lies an opportunity to review and revise the standards even fur¬ 
ther, specifically with regard to a more direct focus on course design. The 
next phase in this on-going study of the iNACOL National Standards for 
Quality Online Courses will be to examine the content validity of the stan¬ 
dards by having experts from various aspects of the field of K-12 online 
learning to provide systematic feedback on the standards themselves, as 
well as the findings from this first phase of validation. 
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